Welcome

 Top Next

Welcome to URL Snooper

version 2.26.01 for Windows 9x/ME/2K/NT/XP/Vista/Win7, December 13th, 2009

Webpage: http://www.donationcoder.com/urlsnooper

URL Snooper was written to help users locate the urls of audio and video files so that they can be recorded.

The goal of URL Snooper is to provide a one-stop easy solution to finding the URLs for all streams. It does this by watching network traffic and identifying potential urls, especially streaming media urls.

Many links to streaming audio and video that you come across on the web are hidden behind javascript or activex scripts. Because of this, it is sometimes very difficult to figure out the actual urls that correspond to the streams being played.

View the complete Version History.


MICROSOFT VISTA AND WIN7 USERS - READ IN

 Top Previous Next

MICROSOFT VISTA AND WINDOWS 7 USERS!

You may have restart your computer after installing urlsnooper and winpcap, and or run the program at least once in administrative mode, before it will detect your adapters and sniff traffic.

To run as administrator you can set this by right clicking on the program file and selecting "Run as Administrator", or going to advanced shortcut properties and permanently setting this value.

I know this is a bit confusing -- see the winpcap page for more help:

http://www.winpcap.org/misc/faq.htm#Q-7


Installation Instructions

URL Snooper Requirements

URL Snooper will work on windows 98,98SE,ME,XP,NT,2K. It normally requires a broadband network connection (cable modem, dsl, ethernet), though it can be used with modem dial-up connections with some effort spent configuring the winpcap driver.

Installation Overview

URL Snooper comes with a standard installation program which will:

· Install the program on your computer
· Install the free WinPCap network driver (required)
· Add the appropriate new group to your start menu
· Create an uninstaller link so you can easily remove all traces of the program should you decide you don't want the program on your computer any more.

You can also choose to unpack the files manually and not use the installer if you wish.

Instructions

To start the installer, double click the URLSnooper.exe file. The installation will begin and walk you through the simple process of installing the program.

URL Snooper uses the excellent and free WinPCap network sniffing driver, which will be installed for you unless you choose to skip this step (you should only skip this step if you already have WinPCap installed, since URL Snooper requires WinPCap in order to work).

Once URL Snooper is installed you can begin using it. You will find it installed in your start menu along with a full help file, or just click the desktop icon.

It's designed to automatically identify the correct network parameters to use on first use so you should be able to begin using it right away without any configuration.


Run it and surf!

URL Snooper operated by default in a "simple" mode which tries hard to make your life easy.

· The first time you run the program it will attempt to automatically detect which of several network adapter drivers works best with your internet connection.
· If there are any problems finding an active network it will be reported to you. Otherwise the program will begin sniffing all network traffic automatically (in simple mode). See the notes on detecting your network adapter.
· Now just visit a website with an audio stream you are interested in and start playing the stream.
· Hit Ctrl+F5 or the Refresh button in your browser to force the page to reload if no results are found, in case the page was "cached" in your computers memory and not fetched from the site recently.

URL Snooper will try to detect the hidden url to the stream and display it in its results list.

When you find a url you are interested in:

· Right-click to perform some operations on it.
· Drag and drop it from URL Snooper onto other programs like Streambox VCR to record them (or copy and paste).

Should you need more advanced control over settings and options, you can switch to Advanced mode from the File menu.


Selecting a Network Adapter

Before you can use URL Snooper, you must select a valid Network Adapter from the Options tab.

There may be multiple adapters available - you will have to try them out until you find the one that monitors traffic on your network.

Follow this procedure until you find the right adapter:

1. Switch to "Options" tab, and select an Adapter from the Drop-down list.
2. Switch to the "Search" tab, and click the "Sniff Network" button.
3. If you get an error, go back to step 1.
4. Otherwise, open up a web page in your browser or perform some other network operation.
5. Watch the statusbar at the bottom of the window. If you see the count of packets increasing, then you have successfully configured your network adapter.
6. If however, the count of packets does not increase as you use the internet, and the Search button is activated, you need to try a different adapter. Repeat step 1.

If none of your adapters work, or if you dont see any network adapters in the drop-down list, then your hardware is not compatible with the network sniffer. Keep your eye on http://winpcap.polito.it/ and on the url snooper forum (http://www.donationcoder.com/urlsnooper) for information about new updates to the WinPcap dlls which may support your hardware.

Some network adapters may not function properly unless the "Only scan this computer" checkbox is checked.

Note: If the packet count on the statusbar increases while you are on the internet, but you aren't seeing any url results, it's probably because you've set the options to only show you Urls related to streaming audio/video. Alteratively, if you are seeing too many urls, you may want to change the option to filter all but streaming urls.


For Fast Learners

Brief Instructions for Fast Learners!

To find the hidden url for a streaming audio/video file:

· Open your browser and go to the page containing the stream you are interested in.
· Start Url Snooper and click on "Sniff Network."
· Go to the browser and start the stream playing.
· Hopefully you will soon see the stream URLs appear in the Results List.
· After the stream begins playing, you may stop it.
· Drag URLs to targets, or right-click to copy to clipboard or launch 3rd party apps.

Selecting the Proper Network Adapter

It can sometimes be tricky to figure out which network adapter entry to use. See the Network Adapter help page in the "How to Use" section.


Getting Started

Before you can use Url Snooper you need to Choose Your Network Adapter.

Finding What You Want

If you want to detect URLs on a web page, click "Sniff Network" on the Main Search tab, and then open your browser and go to the page in question. The discovered URLs will appear in the results list. Selecting a found entry will display the raw binary bytes surrounding the found URL. A right-click will let you copy the URL to the clipboard, or you can drag it into a URL-aware application (like a web browser).

If you want to detect the URL for a hidden stream, click "Sniff Network" to begin network sniffing, then go to the page and start the stream playing. The results list will show all URLs found. Some trial and error may be required to figure out which URL corresponds to the stream you are looking for.


Options

The "Options" tab allows you to adjust some common options for the application.

Network Sniffing Options

· Clear Results Before New Search - This will clear any existing results when a new search is started.
· Remove Duplicates - Don't list urls more than once even if found multiple times.
· Only scan packets for this computer - The application can sniffing either every packet that passes through the network adapter (including to and from other machines on your network), or just those coming to and from your local computer. NOTE: some intel network adapters may only work if this box is checked.
· Non-default Browser - If specified, this browser will be used to launch urls and websites from the Website menu.
· Sniffing Engine Driver - Currently only the WinPcap engine is supported, but other engines may be supported in the future.
· Network Adapter - This is where you select the network adapter to sniff.
· External Url Result Links - See this help topic.

screenshot


Filtering

Filtering

There may be many many urls found while surfing or searching; to help you limit the url list to those urls of interest, you can tell Url Snooper to filter out urls that do not correspond to multimedia or streaming media resources. You can change this option at any time to temporarily hide non matching urls.

Some urls have additional information associated with them. Often this information is not useful for downloading, and you can use the filter drop down selection to hide this extra information.

You can also specify a keyword filter to help you quickly narrow down the found urls; just clear this box to show all.

Separate multiple keywords with |

Use - in front of a keywords to HIDE any urls with that keyword

Also search for

Specifying extra terms in the "Also Search For" box will also find the occurence of the spcified terms, even in the absense of urls.


The Result List

Search results are shown in the Result List.

You can drag an entry from this list to another program. The entry is treated as a URL, and so will only work with target programs that can accept URLs.

You can right-click in the results window to copy an entry to the clipboard, or launch an external program specified in the External Links Options.

You can also save found results to a text file using the File Menu.


Customizing External Links

External Links

To help make using the application more convenient, URL Snooper can be configured to launch external 3rd party applications in a flexible way.

You add external links to 3rd party program at the bottom of the "Options" tab. Each entry speficies the location of a program, the commandline arguments to pass, and the text to use for the pop-up menu.

Menu text for the links will appear when you right-click in the Results List (you can use the standard windows protocol of placing an ampersand before a letter in the text to let the user select the item with a keypress).

Rather than delete entries that are inappropriate, you can simply uncheck them, which will prevent them from being included in the Result List pop-up menu.

Adding and Modifying Entries

Right-click in the list of external links to add, delete, or modify an entry.

When specifying commandline arguments, you can use keywords which will be replaced by URL Snooper when the program is launched:

$url url string

$file full filename (with path) of file containing the url

$dir directory where file containing the url resides

$offset offset into file where url string begins (in decimal)

$offseth offset into file where url string begins (in hex)

$length length of url in bytes


Customizing the Favorites Menu

You can customize the list of websites in the "Favorite Websites" menu. Just edit the UserWebsites.txt file in a normal text editor.


Searching Files

File Searching

You may want to search files for a variety of reasons. If you want to search the temporary internet files that get created as you browse the web. The files being searched are listed on the StatusBar to help you judge progress. If you are trying to locate the URL of a hidden stream using the file search method, do so by conducting the search after you have begun to play the stream in question.

You can access the file searching functions throuh the File menu, or you can drag and drop files and directories onto the Url Snooper window to have them searched immediately.

File Searching Options

The key to fast file searching is to exclude files that you know won't contain what you're looking for. You can limit the file search by directory, file size, and file modification date. Limits based on file extension are coming soon. File searching can also be used to find hidden urls inside exe files and any other file.

· File modification filter - You can limit search to only recent files. This can be especially useful for avoiding the display of large numbers of urls which you have already found and which you know do not contain the information you are looking for. The "Since last search" option keeps track of when you performed your last search, and only searches newer files; note that you may want to de-select this option if you change other options, since there may be older files that match the new criteria.
· File size filter - One of the most effective filters you can use is the file size filter. Huge files are typically binary files and executables which will not contain the text you are searching for. Excluding such files can dramatically speed up search time.
· Directories to search - If you are looking for temporary internet files, Internet Explorer keeps them in C:\Windows\Temporary Internet Files\. Where does Netscape keep them?


Windows VISTA - Troubleshooting Notes

Windows VISTA is still in beta at the time of this release. We expect further releases of this program before VISTA is officially released, but this program can be used in VISTA with a little bit of tweaking:

Set the compatibility mode on the URL Snooper executable to run as an administrator and check the campatibility option for XP SP2. You may also want to visit the WinPCap site and check for an update of the WinPCap driver for Windows VISTA: http://www.winpcap.org/.


Detecting your Network Adapter

URL Snooper now has some advanced routines to automatically detect the network adapter to use to sniff network traffic.

It detects it by requesting a page from the internet and checking network adapter drivers till it finds one that it observes processing the data.

If the automatic network detection fails, it might be because your firewall is blocking the test page request - if so, make sure you allow it. Or perhaps your network connection is down at the time you try to connect.

You can disable the automatic network detection process and manually select a network adapter from the Advanced mode Options tab.

If you cannot find a network adapter that works, you might have one of the very few network adapters that the WinPCap system has trouble with.. See http://www.winpcap.org/contact.htm for more help and guidance.


Sniffing works but can't find the URL

URL Snooper isn't magic - it detects URLs by looking for the telltale url links in network traffic.

Sometimes it can get fooled by non-standard URL links on pages.

If it can't find a URL on a page, as a last resort you can switch to advanced mode and select "All URLs" from the drop down filter list, and try to identify the non-standard URL. You can also look at the surrounding context to get a better idea of the problem.


If you still can't figure it out

If you're still stuck:

Try the DonationCoder.com forum and ask for help on the URL Snooper section: http://www.donationcoder.com/Forums/bb/index.php?board=17.0


Right-click Crash - Conflict with "Mmm" Tool

We have reliable reports that the context menu tool "Mmm" can conflict with Snooper and cause crashes when you right-click on a URL.


Version History and To Do List

Version History

Version 2.26.01 - December 13, 2009

· [BugFix] Improved detection of network adapters -- was failing to list all available adapters on some people's computers. Thanks to ixxo and everyone else on dc forum who reported and helped test (see here for the thread).

Version 2.25.01 - October 29, 2009

· Updated with latest version of WinPCap (4.1.1) - Now supports Windows 7
· [BugFix] The error about index out of range when network adapters have changed on restart should be fixed.
· [BugFix] Settings file was still being saved in application directory even on Win Vista/7; new version respects configdir.ini and portable settings, as well as storing working files in user documents directory by default on Win Vista/7.

Version 2.23.01 - March 14, 2009

· Updated with latest version of WinPCap (4.1beta5).
· Improved wording of automatic network adapter detection failure message.
· Improved packet reconstruction should reduce chance of packets being split in the middle.
· [BugFix] Urls incorrectly shown as from http://0.0.0.0 fixed.
· Added SourceIP column.

Version 2.22.02 - December 24, 2008

· [BugFix] Default URL for network adapter testing was a google page that no longer exists; updated.
· [Change] URL Snooper should now insist on Admin rights when be running on Vista.

Version 2.22.01 - June. 4, 2008

· Added option to customize the url used to test network adapter auto detection, changed default from donationcoder to small google css stylesheet page (this takes load off donationcoder server and should reassure concerned that urlsnooper was somehow "phoning home").
· Fixed language translation files.
· Increased max URLS that can be kept at once.
· New option to discard filtered-out urls that aren't display; normally the program stores all urls found even if its currently set to filter out all but multimedia urls, so that you can change the view filter to show all urls after the fact. The new option will discard them during filtering, which is a good solution for people who were ending up storing huge numbers of urls while searching for multimedia urls.
· Hopefully fixed "Menu - Index out of range" error.
· Removed some code to reassemble packets (it's possible this could result in worse url grabbing in rare cases -- stay tuned).
· Upgraded to winpcap 4.1beta4.

Version 2.21.01 - Dec. 12, 2007

· Increased packet buffer size, should eliminate error on some computers where it reports "Packets are bigger than reserved buffer" (see here).
· Upgraded to winpcap 4.1beta2
· Added back buttons to parse single packets.

Version 2.19.02 - Aug. 18th, 2007

· Fixed bug which was causing crashes win win9x/ME
· Upgraded to winpcap 4.1beta
· Added update support
· Changing languages was messing up the network adapter combo box.
· Changing network adapter manually now disables the autonetwork detection checkbox.
· Added instructions and settings so that the program runs with Administrator Privileges in Microsoft Vista.

Version 2.18.01 - Feb. 4th, 2007

· Updated installer to use WinPcap 4.0 final on WinXp/Nt/Vista
· Added code so installer uses WinPcap 4beta2 on Win9x/Me (last version to support these versions)

Version 2.16.01 - Nov. 30th, 2006

· Fixed tray icon
· Now works on Windows VISTA (!)

Version 2.15.01 - Nov. 28th, 2006

· Snooper was unnecessarily checking website when snooping even when autodetect network adapter was not set - thanks DPK.

Version 2.14.02 - Nov. 6th, 2006

· Updated to properly autodetect network adapter when IE7 is installed.

Version 2.13.04 - 7/18/06

· Added detection of some multimedia keywords (WMA,MP3) in URL arguments as potential multimedia URLs
· Added .asx as multimedia URL extension
· Minor changes to help file and manual
· Added troubleshooting note about an error if you skip installing WinPCap.
· Various spelling errors corrected.
· Added a couple more troubleshooting help pages

Version 2.10.06 - 7/02/06

· Detects and identifies "reflector:" links as streaming media links.

Version 2.09.10 - 6/22/06

· Updated help file.
· Sped up changing of filter display
· If you disabled autotesting for network card it could sometimes retest until a restart of program.
· Improved inferencing of ip address of partial (relative) URLs.
· Improved multi-packet assembling.
· Added notes for installing on Windows VISTA.

Version 2.09.08 - 6/20/06

· Installer makes clearer that WinPCap install is mandatory.
· Fixed rare but harmless "index out of range" error on switching to Advanced mode.
· Fixed rare but bug where mouse click was ignored on buttons.

Version 2.08.02 - 6/10/06

· Very minor text changes in help file and installer.
· File+View menus combined.
· Favorite websites menu removed from simple mode.

Version 2.06.05 - 5/07/06

· Improved handling if a language file is deleted.
· Improved some multilingual features.
· Thanks to Joto (German Translation)
· Thanks to Simeon Lapinbleu and Vianney Jonville (French Translation)

Version 2.06.04 - 5/02/06

· Made Simple mode default.

Version 2.06.03 - 5/02/06

· Major feature: Added simple and advanced (normal) modes; simple is much easier for new users.
· Major feature: Added Automatic network adapter selection - it will now test all adapters until it finds one that works and can sniff URLs.
· Major feature: Multilingual support.
· Minor feature: Minimize to tray button.
· Minor feature: Added "flv" to multimedia list
· Minor feature: Added "edit favorites" item in menu.
· Minor improvement: Added http:// to relative urls that need them.

Version 2.05.01 - 12/9/05

· no more separate manual url download dialog - you can down type in a url directly at bottom of form to EITHER download and scan source code, or start network searching and sniff traffic; should make it easier for quick jobs.
· new license policy - program will never stop working only nag for you to get a renewed freeware key after 6 months, and after 1 year you can download full permanent non-expiring license key.
· fixed tab ordering of some controls.
· there are now 2 save urls options (save all and save selected)
· quick filter is now colored red to prevent you from forgetting you have it active
· save url file no longer has extra \r at end of each line.
· file searching was not returning case sensitive urls
· added .pls to multimedia extensions
· added ability to use - in front of filter keywords to say hide strings with keyword

· Version 2.04.04 - 10/03/05

1) the method turning off promiscuous mode has been adjusted to better work around bugs preventing some lan adapaters (INTEL) from sniffing unless promiscous mode is turned off in a specific fashion. If your adapter doesn't seem to sniff traffic make sure to try checking "only from this computer" option.

2) the only sniff from this computer option has been made default.

· Version 2.04.01 - 5/15/05

1) urls were being forced to lowercase - fixed

· Version 2.03.09 - 3/10/05

1) New customizable Websites menu.

2) Detection of more url link types, better reporting.

3) Ability to specify custom browser.

4) Packet spanning(!) to avoid split urls.

5) Now hosted officially at http://www.donationcoder.com.

6) New Always-On-Top option

7) New Help file.

· Version 2.01.00 - 02/06/05

1) version 2 marks a major overhaul to the user interface in order to simplify it and make operation easier.

2) more flexible filtering options.

3) better removal of duplicates.

4) better identification of relative links.

5) better identification of protocol information.

· Version 1.02.01 - 3/17/03

1) Compiled and packaged with WinPcap library v3.0 (http://winpcap.polito.it/).

2) Restructured the packet-viewing interface; now instead of a separate window, packets are shown in normal packet view if "View Live Packets" is checked.

3) New checkbox to hide file date/path/name columns, useless when doing network sniffing.

4) Added detection and rebuilding for relatively addressed files making use of "Host: " and "Get \" fields

· Version 1.1 beta 1 - 06/10/02

1) Improved help for selecting the right network adapter.

2) Support for new WinPcap library v2.3 and later, which should provide better Windows XP support (http://winpcap.polito.it/).

3) All users may now select whether to restrict sniffing to current computer on a lan (only works for Win9x/ME ?).

4) Built in raw packet viewer for watching ALL packets.

5) fixed bug where extra ' character at end of url was not removed (thanks to finkl).

· Version 1.0 beta 6 - 02/21/01

1) Fixed minot bug in keyword searching that would result in blank url captions for found keywords.

2) Fixed small bug where the last character of a sream was a \ which resulted in an error if you tried to paste into streambox.

3) First public release of program.

· Version 1.0 beta 5 - 02/17/01

1) MAJOR: Turns out the new network drivers were cutting off packets at 255 bytes maximum length, which resulted in URL Snooper not detecting many urls. Fixed.

2) Fixed bug where urls were automatically lowercased, resulting in some urls not being downloadable.

3) Confirmation (thanks bovric) that URL Snooper works on Windows ME.

4) Increased cpu time devoted to consuming packets in the local buffer, should result in less buildup of packets, possible slower gui response during periods of high network traffic.

· Version 1.0 beta 4 - 02/17/01

1) Fixed problem with filtering local traffic on computers with multiple ips.

2) Added ability to use previous Distinct Network drivers via a registry

setting (see AdvancedNetworkDriverInfo.txt).

3) Windows NT and Windows 2000 compatibility has been confirmed.

4) Modified the installer so that it automatically launches the WinPcap

network driver installation program.

· Version 1.0 beta 3 - 02/15/01

1) Another try at Win2000 compatibility.

2) Installation now automatically launches WinPcap network driver installer.

3) Now forces rename of initial default config file to avoid overwriting

this file during upgrades.

· Version 1.0 beta 2 - 02/13/01

1) MAJOR: Changed to network drivers which should work on win 2000 and win nt!

2) URL comment heuristic now also scans file extensions (.asf,.wma,.ra,etc.).

"Hide Non-streaming URLs" option will no longer hide urls of the form

"http://www.site.com/file.asf"

3) Fixed bug where if url ended at the end of a file or packet, last

character was truncated.

4) Fixed bug where network packet text was being forced to lowercase

5) Fixed the about box to indicate the proper program version.

· Version 1.0 beta 1 - 02/13/01

1) First Release of program

Here are some things that being considered for future releases:

· multilingual stuff - delete all .XXX when the .XXX.lng is newer
·
· add ndis driver mode
· move to a generic, configurable regex checklist for extracting urls
· some urls (for example see cspan.org) have '; left on at end of them.
· Fix save changes bug, wont let you save over default.cfg
· Save registry to Current User instead of Local Machine
· Ability to configure which fields to show in result list (or at least hide silly ones)
· Ability to specify file extension filters.
· More flexible and general pattern searching.
· More helpful comments for located items.

· smart threaded testing of urls
· smarter quick rejection of urls
· better protocol labeling for pls urls
·
· fix resize column nextgrid bug
· check for preset value in preset form set


Credits and Thank You's

We want to thank to everyone at the Streambox VCR Underground and the associated discussion forum. Especially thanks to the beta testers there who are helping us improve the program (especially veblin).

WinPcap is the awesome free network sniffing code library that we use; thanks to the wonderful work done by the WinPcap team (http://winpcap.polito.it/) - without them, URL Snooper on Windows 2000 would not have been possible.

Lastly, I need to thank those kind users who have taken the time and effort to donate at http://www.donationcoder.com. Your donations have made continued work on this program possible. Thank you.

Thank you to the translators:

· Joto (German Translation)
· Simeon Lapinbleu and Vianney Jonville (French Translation)


License

 Top Previous

This program is freeware; you may use it as you wish, but only for non-commercial purposes.

It's development is funded by volunteer user donations at http://www.donationcoder.com.